Incremental port #1322

sheetalkamat · 2025-06-30T18:28:19Z

No description provided.

… as for better readability of baselines

// Options are always detected as changed so everything is refreshed on incremental build

Copilot

Pull Request Overview

This PR continues the incremental port of the TypeScript compiler with baseline test updates reflecting changes to output formatting, command-line invocation, and internal processing. The changes primarily update test baselines to match the new compiler's behavior patterns.

Key changes:

Standardized test output formatting to use tsgo command instead of tsc
Updated baseline outputs to include library references and diagnostic information
Removed references to deprecated outFile option and related configuration errors

jakebailey

Seemingly still logical races or something; note the CI failure:

Using D:\a\typescript-go\typescript-go\Herebyfile.mjs to run baseline-accept
Starting baseline-accept
Finished baseline-accept in 19ms
Completed baseline-accept in 20ms
diff --git a/testdata/baselines/reference/tsc/incremental/when-global-file-is-added,-the-signatures-are-updated.js b/testdata/baselines/reference/tsc/incremental/when-global-file-is-added,-the-signatures-are-updated.js
index e3612043..f4adbdf5 100644
--- a/testdata/baselines/reference/tsc/incremental/when-global-file-is-added,-the-signatures-are-updated.js
+++ b/testdata/baselines/reference/tsc/incremental/when-global-file-is-added,-the-signatures-are-updated.js
@@ -551,7 +551,6 @@ declare function something2(): number;
 //// [/home/src/workspaces/project/src/fileNotFound.js] *new* 
 function something2() { return 20; }
 
-//// [/home/src/workspaces/project/src/main.js] *modified time*
 //// [/home/src/workspaces/project/tsconfig.tsbuildinfo] *modified* 
 {"version":"FakeTSVersion","fileNames":["../../tslibs/TS/Lib/lib.d.ts","./src/filePresent.ts","./src/fileNotFound.ts","./src/anotherFileWithSameReferenes.ts","./src/newFile.ts","./src/main.ts"],"fileInfos":[{"version":"7dee939514de4bde7a51760a39e2b3bfa068bfc4a2939e1dbad2bfdf2dc4662e-/// \u003creference no-default-lib=\"true\"/\u003e\ninterface Boolean {}\ninterface Function {}\ninterface CallableFunction {}\ninterface NewableFunction {}\ninterface IArguments {}\ninterface Number { toExponential: any; }\ninterface Object {}\ninterface RegExp {}\ninterface String { charAt: any; }\ninterface Array\u003cT\u003e { length: number; [n: number]: T; }\ninterface ReadonlyArray\u003cT\u003e {}\ninterface SymbolConstructor {\n    (desc?: string | number): symbol;\n    for(name: string): symbol;\n    readonly toStringTag: symbol;\n}\ndeclare var Symbol: SymbolConstructor;\ninterface Symbol {\n    readonly [Symbol.toStringTag]: string;\n}\ndeclare const console: { log(msg: any): void; };","affectsGlobalScope":true,"impliedNodeFormat":1},{"version":"8f597c315d3bba69a79551601042fdcfe05d35f763762db4908b255c7f17c7d5-function something() { return 10; }","signature":"4f3eeb0c183707d474ecb20d55e49f78d8a3fa3ac388d3b7a318d603ad8478c2-declare function something(): number;\n","affectsGlobalScope":true,"impliedNodeFormat":1},{"version":"6c5b229cbf53b2c6867ab14b139eeac37ed3ec0c1564ba561f7faa869aaba32c-function something2() { return 20; }","signature":"14ba6a62cd6d3e47b343358b2c3e1b7e34b488b489f0d9b915c796cd2e61bbad-declare function something2(): number;\n","affectsGlobalScope":true,"impliedNodeFormat":1},{"version":"552b902790cfbb88a7ed6a7b04b24bb18c58a7f52bcfe7808912e126359e258d-/// \u003creference path=\"./filePresent.ts\"/\u003e\n/// \u003creference path=\"./fileNotFound.ts\"/\u003e\nfunction anotherFileWithSameReferenes() { }","signature":"d4dbe375786b0b36d5425a70f140bbb3d377883027d2fa29aa022a2bd446fbda-declare function anotherFileWithSameReferenes(): void;\n","affectsGlobalScope":true,"impliedNodeFormat":1},{"version":"a4c88e3619994da0f5e4da2dc210f6038e710b9bb831003767da68c882137fb1-function foo() { return 20; }","signature":"f0d67d5e01f8fff5f52028627fc0fb5a78b24df03e482ddac513fa1f873934ee-declare function foo(): number;\n","affectsGlobalScope":true,"impliedNodeFormat":1},{"version":"930f3c7023d87506648db3c352e86f7a2baf38e91f743773fddf4a520aff393a-/// \u003creference path=\"./newFile.ts\"/\u003e\n/// \u003creference path=\"./filePresent.ts\"/\u003e\n/// \u003creference path=\"./fileNotFound.ts\"/\u003e\nfunction main() { }something();something();foo();","signature":"ed4b087ea2a2e4a58647864cf512c7534210bfc2f9d236a2f9ed5245cf7a0896-declare function main(): void;\n","affectsGlobalScope":true,"impliedNodeFormat":1}],"fileIdsList":[[2,3],[2,3,5]],"options":{"composite":true},"referencedMap":[[4,1],[6,2]],"latestChangedDtsFile":"./src/fileNotFound.d.ts"}
 //// [/home/src/workspaces/project/tsconfig.tsbuildinfo.readable.baseline.txt] *modified*

jakebailey · 2025-07-21T22:49:37Z

internal/ast/diagnostic.go

@@ -21,6 +21,7 @@ type Diagnostic struct {
 	relatedInformation []*Diagnostic
 	reportsUnnecessary bool
 	reportsDeprecated  bool
+	skippedOnNoEmit    bool


Do the other functions that compare/do equality need to be updated to check this?

jakebailey · 2025-07-21T22:54:03Z

internal/collections/set.go

+	if s.Len() != other.Len() {
+		return false
+	}
+	for key := range s.M {
+		if !other.Has(key) {
+			return false
+		}
+	}


I think you could probably just use maps.Equal(s.M, other.M).

jakebailey · 2025-07-21T22:54:28Z

internal/collections/syncset.go

+	})
+}
+
+func (s *SyncSet[T]) ToArray() []T {


I'd say ToSlice() here, just since it's not actually a Go array.

jakebailey · 2025-07-21T22:55:18Z

internal/compiler/emitter.go

@@ -462,3 +469,93 @@ func getDeclarationDiagnostics(host EmitHost, file *ast.SourceFile) []*ast.Diagn
 	transform.TransformSourceFile(file)
 	return transform.GetDiagnostics()
 }
+
+type AnyProgram interface {


Is there a better name for this than "AnyProgram"?

jakebailey · 2025-07-21T22:56:24Z

internal/compiler/host.go

@@ -72,6 +77,9 @@ func (h *compilerHost) GetSourceFile(opts ast.SourceFileParseOptions) *ast.Sourc
 }

 func (h *compilerHost) GetResolvedProjectReference(fileName string, path tspath.Path) *tsoptions.ParsedCommandLine {
-	commandLine, _ := tsoptions.GetParsedCommandLineOfConfigFilePath(fileName, path, nil, h, nil)
+	if h.extendedConfigCache == nil {
+		h.extendedConfigCache = &collections.SyncMap[tspath.Path, *tsoptions.ExtendedConfigCacheEntry]{}


This seems like it is concurrency unsafe. But, the coverage never says this happens, so maybe this should just panic?

jakebailey · 2025-07-21T23:01:53Z

internal/execute/tsc.go

@@ -41,25 +41,20 @@ func applyBulkEdits(text string, edits []core.TextChange) string {
 	return b.String()
 }

-func CommandLine(sys System, cb cbType, commandLineArgs []string) ExitStatus {
+func CommandLine(sys System, commandLineArgs []string, testing bool) (ExitStatus, *incremental.Program, *Watcher) {


Is it time to start returning a struct here? I assume that this is all just here because testing needs more info out of the CommandLine function?

jakebailey · 2025-07-21T23:05:47Z

internal/compiler/emitter.go

+		// Options diagnostics include global diagnostics (even though we collect them separately),
+		// and global diagnostics create checkers, which then bind all of the files. Do this binding
+		// early so we can track the time.
+		bindStart := recordTime("bind", true, time.Time{})


I don't know how I feel about this pattern... Feels a bit weird for the emitter file to be the one with this time tracking code, and then it be indiredcted through callbacks and so on, when it was quite direct before. What was wrong with the old way? Can this just return a function closure that is used to start/stop the range?

jakebailey · 2025-07-21T23:07:23Z

internal/execute/tsc.go

+func emitAndReportStatistics(
+	sys System,
+	program compiler.AnyProgram,
+	getCoreProgram func() *compiler.Program,


Is this indirection needed? Can't just be immediately invoked? Or just use program?

jakebailey · 2025-07-21T23:11:18Z

internal/incremental/emitfileshandler.go

+		if h.program.updatedSignatureKinds != nil {
+			h.program.updatedSignatureKinds.Store(file, SignatureUpdateKindStoredAtEmit)
+		}
+		h.program.snapshot.buildInfoEmitPending = true


I haven't gotten there yet, but it strikes me as odd that we are atomically mutating one thing at the same time as we are not atomically mutating another thing, when both are sort of "on" the same Program... I can't really put my finger on it but it gives me the bad feeling of the model not being consistent and free of logical races.

jakebailey · 2025-07-21T23:14:32Z

internal/tsoptions/enummaps.go

@@ -217,6 +217,11 @@ var targetToLibMap = map[core.ScriptTarget]string{
 	core.ScriptTargetES2015: "lib.es6.d.ts", // We don't use lib.es2015.full.d.ts due to breaking change.
 }

+// Used for test to add the files


I would just delete this; I initially read this as it saying that a test was modifying the map, when it's just adding fake files.

That being said, why do we need this when the harness already knows how to do this without this mapping?

sheetalkamat force-pushed the incremental branch 7 times, most recently from 66a62bf to 04b68cb Compare July 3, 2025 22:03

tmm1 mentioned this pull request Jul 10, 2025

High CPU usage with --watch when idling on MacOS #1317

Open

sheetalkamat force-pushed the incremental branch from 04b68cb to 961ab06 Compare July 21, 2025 21:46

sheetalkamat added 21 commits July 21, 2025 14:47

Refactor

ede403a

Add stub for incremental

ba71811

Incremental

5910a23

Make buildinfo tests portable and readable

b642e22

Handle casing with compiler options serialization

4ef2a94

make common js default impolied file format

da2a0c4

More changes

9123231

More changes

6c0f24e

fix test

b6cdfce

more change

a2f13d9

Parallel!!!

703cd4b

Tests

4ad58ed

Watch changes

7395f27

Verify diagnostic and options serializing

77c8a12

Tests for reference map

31a8c76

Lint

3f2fbfd

Use fake lib contents to make it deterministic for baselining as well…

a6cdc84

… as for better readability of baselines

Fix race with latest dts file

93c6232

Fix more tests

bdb9076

change to test

8c0d4b5

Lint

8c62de6

sheetalkamat added 16 commits July 21, 2025 14:47

Support for edits in tsc tests

f93f053

Fix race with default libs

efd838e

Always collect references for proper program update

1973134

Add baseline for semantic diagnostics refresh to check for correctness

2ab1229

No need for scenario name in test name

8117de5

Baseline signature compute

30f10d3

Test incremental edit

3a5e3ac

// Options are always detected as changed so everything is refreshed on incremental build

Fix modified edit baseline

e1b3dc7

Use semantic diagnosics from old snapshot to compare

e7df40d

Fix incremental updates

766d372

More tests and updates

d0cefe6

Unify tsc runner even more

e048d22

Test fs refactor

2f89867

rename test baselines

776ba84

Write signature and its text for testing

8d8c62c

Incremental correctness

f4d5056

sheetalkamat force-pushed the incremental branch from 961ab06 to f4d5056 Compare July 21, 2025 22:09

Cleanup

7d7d4f8

sheetalkamat requested review from weswigham, andrewbranch and jakebailey July 21, 2025 22:38

sheetalkamat marked this pull request as ready for review July 21, 2025 22:38

Copilot AI review requested due to automatic review settings July 21, 2025 22:38

Copilot AI reviewed Jul 21, 2025

View reviewed changes

Fix lint

953be0d

jakebailey reviewed Jul 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Incremental port #1322

Incremental port #1322

sheetalkamat commented Jun 30, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

jakebailey left a comment

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

jakebailey Jul 21, 2025

Uh oh!

Uh oh!

Incremental port #1322

Are you sure you want to change the base?

Incremental port #1322

Conversation

sheetalkamat commented Jun 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

jakebailey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!